TITLE OF THE INVENTION 



APPARATUS AND METHOD FOR ADAPTIVE MOTION COMPENSATED 
DE-INTERLACING OF VIDEO DATA 

CROSS REFERENCE TO RELATED APPLICATIONS 

[0001] This application claims the benefit of Korean Application No. 2001-32494 filed 
June 11, 2001, in the Korean Patent Office, the disclosure of which is incorporated herein by 
reference. 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

[0002] The present invention relates to an apparatus for and a method of converting a 
format of a video signal, and more particularly, to an apparatus for and a method of adaptive 
motion compensated de-interlacing of video data, in which either a spatio-temporal method 
of interpolation or a motion compensation method of interpolation is adaptively selected 
according to a motion type. 

2. Description of the Related Art 

[0003] Interlaced scanning in which two fields form a single frame is usually applied to a 
television video signal. To display interlaced video on a screen of a recent personal 
computer (PC) or high definition television which displays images usually using progressive 
scanning, it is necessary to generate video lines to fill blank lines using a de-interlacing 
method. 
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[0004] FIG. 1 is a basic conceptional diagram of typical conventional de-interlacing. 
Referring to FIG. 1, de-interlacing is used to convert a field including odd or even samples in 

a vertical direction into a frame. An output frame F 0 (x,n) is defined as Equation (1). 



i7 r \ F ( X > R )> & mod2 = n mod2 )> 

F 0 (x,n) = < _ ...(1) 
\F t (x,n), otherwise 



Here, x denotes a spatial position, n denotes a field number, F{x,n) denotes an input 

field, and F t (x,ri) denotes an interpolated pixel. In the expressions y mod 2 and n mod 2, 

"mod" represents a module operation which gives a remainder after a division. For example, 
5 mod 2=1,4 mod 2 =0, 3 mod 2 = 1 and 2 mod 2 = 0. 

[0005] FIG. 2 shows a 3 pixel x 3 pixel window in which a conventional Edge-based Line 
Averaging (ELA) de-interlacing algorithm which does not use motion compensation is 
applied. Referring to FIG. 2, ELA de-interlacing uses correlation between pixels with regard 
to a directivity (x,y) at a position of a pixel to be interpolated, as shown in Equation (2). In 
other words, an average value of pixels to be interpolated is obtained using pixels adjacent 
to the pixels to be interpolated. 



F 0 {x,n) = 



F(x-u x -u y ,ri) + F(x + u x -u y ,n)/2, if a<b and a<c, 
F(x-u x +u y ,n) + F(x + u x -u y ,n)/2, if b>a and b<c, ...(2) 
F(x - u y ,n) + F(x + u y ,n)/2, (otherwise) 



Here, a = \F(x-u x -u y ,n)-F(x+u x +u y ,n)\ 
b = |f(x - u x +u y ,n)-F(x + u x -u y ,ri)\ 
c — \f(x — u* y , ri) — F(x + u* y , n)\. 
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[0006] FIG. 3 is a conceptional diagram of a typical conventional time-recursive (TR) 
de-interlacing method. Referring to FIG. 3, in TR de-interlacing using a motion vector, it is 
assumed that an («-1)-th field is completely de-interlaced, and missing data of an «-th field is 
compensated for using a motion vector. A sample to be interpolated may be an original 
sample in the )-th field or an interpolated sample in the )-th field. Accordingly, the 
sample to be interpolated can be expressed by Equation (3). 



[0007] However, since an ELA de-interlacing method does not use motion compensation, 
flickering and blurring occur in an area in which there is motion. Since a TR de-interlacing 
method performs de-interlacing consecutively, an error in one field may be propagated to 
another field. Further, a conventional motion compensation method has a disadvantage of 
generation of image artifacts. 

SUMMARY OF THE INVENTION 

[0008] To solve the above-described and other problems, it is a first object of the present 
invention to provide a de-interlacing method of improving picture quality by adaptively 
selecting either a spatio-temporal method of interpolation or a motion compensation method 
of interpolation according to a motion type. 

[0009] It is a second object of the present invention to provide a de-interlacing apparatus 
which adaptively selects either spatio-temporal de-interlacing or a motion compensated 
de-interlacing according to a motion type. 

[0010] Additional objects and advantages of the invention will be set forth in part in the 
description which follows, and, in part, will be obvious from the description, or may be 
learned by practice of the invention. 
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[001 1] To achieve the above and other objects of the invention, there is provided a 
method of converting an interlaced video signal to a progressive video signal. The method 
comprises estimating a motion vector of a field to be interpolated and an accuracy of the 
motion vector using a motion vector between fields, determining existence or non-existence 
of global motion by analyzing the estimated motion vector, generating a pixel to be 
interpolated by a first method which performs motion compensation with respect to the field 
to be interpolated using the estimated motion vector, generating the pixel to be interpolated 
by a second method using pixels adjacent to the pixel to be interpolated in the field to be 
interpolated and pixels adjacent to the pixel to be interpolated in fields preceding and 
succeeding the field including the pixel to be interpolated, and selectively outputting either 
the pixel generated by the first method or the pixel generated by the second method 
according to the estimated accuracy of the motion vector and the determined existence or 
non-existence of global motion. 

[0012] To achieve the above and other objects of the invention, there is also provided an 
apparatus which converts an interlaced video signal to a progressive video signal. The 
apparatus comprises a motion estimation unit which estimates a motion vector of a field to 
be interpolated using a motion vector between fields, a motion decision unit which 
determines existence or non-existence of global motion by analyzing the motion vector 
estimated by the motion estimation unit, a motion compensated interpolation unit which 
performs motion compensation with respect to the field to be interpolated using the motion 
vector estimated by the motion estimation unit to generate a pixel to be interpolated, a 
spatio-temporal interpolation unit which generates a pixel to be interpolated using pixels 
adjacent to the pixel to be interpolated in the field to be interpolated and pixels adjacent to 
the pixel to be interpolated in fields preceding and succeeding the field including the pixel to 
be interpolated, and an adaptive selection unit which selectively outputs either the pixel 
generated by the motion compensated interpolation unit or the pixel generated by the 
spatio-temporal interpolation unit according to the existence or non-existence of global 
motion determined by the motion decision unit. 
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[0013] The above objects and advantages of the present invention will become more 
apparent by describing in detail an embodiment thereof with reference to the attached 
drawings in which: 

FIG. 1 is a basic conceptional diagram of typical conventional de-interlacing of video 

data; 

FIG. 2 shows a 3 pixel by 3 pixel window in which a typical conventional 
de-interlacing algorithm is applied; 

FIG. 3 is a conceptional diagram of a typical conventional time-recursive (TR) 
de-interlacing method; 

FIG. 4 is a block diagram of a de-interlacing apparatus according to an embodiment 
of the present invention; 

FIG. 5 is a detailed flowchart of the operation of the motion analysis unit in FIG. 4; 

FIG. 6A is a diagram illustrating motion vectors of a current scene having global 

motion; 

FIG. 6B is a histogram of occurrences of motion vectors of adjacent homogeneous 
fields corresponding to the current scene; 

FIG. 6C is a diagram illustrating motion vectors of another video image; 

FIG. 6D is a histogram of occurrences of the motion vectors of FIG. 6C; 

FIG. 7 is a detailed flowchart of the operation of the motion type determination unit 
shown in FIG. 4; and 

FIG. 8 is a detailed flowchart of operation of the adaptive selection unit of FIG. 4. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0014] Reference will now be made in detail to the present embodiments of the present 
invention, examples of which are illustrated in the accompanying drawings, wherein like 
reference numerals refer to like elements throughout. 

[001 5] FIG. 4 is a block diagram of a de-interlacing apparatus according to an 
embodiment of the present invention. Referring to FIG. 4, a motion estimation unit 410 
estimates motion vectors of an «-th field to be interpolated from («-1)-th and (n+1)-th fields 
and estimates an accuracy of each motion vector. Here, the motion estimation unit 410 



estimates the motion vectors between adjacent fields having the same properties and 
applies half of the motion to the field to be interpolated between the adjacent fields. A cost 
function or a Sum of Absolute Difference (SAD) is used to indicate the accuracy of each 
motion vector in the field to be interpolated. The SAD of pixels of a block is accumulated 
and used to obtain a value for calculating the accuracy of the motion vector. 

[0016] A motion information decision unit 420 includes a motion analysis unit 422 and a 
motion type determination unit 424. The motion information decision unit 420 analyzes the 
motion vectors estimated by the motion estimation unit 410 and determines a motion type. 
More specifically, the motion analysis unit 422 analyzes the motion vectors estimated from a 
current scene or field to estimate existence/non-existence of global motion and to estimate a 
direction of the global motion. The motion type determination unit 424 determines a motion 
type (global motion or local motion) of each block referring to the motion vectors estimated 
by the motion estimation unit 410 and the global motion of a scene analyzed by the motion 
analysis unit 422. 

[0017] A motion compensated interpolation unit 440 generates each pixel to be 
interpolated from adjacent field images using a corresponding motion vector estimated by 
the motion estimation unit 410. 

[0018] A spatio-temporal interpolation unit 450 generates each pixel to be interpolated 

using pixels adjacent to the pixel to be interpolated in a field to be interpolated ( the 

n-th field) and pixels adjacent to the pixel to be interpolated in the («-1)-th and («+1)-th fields. 

[0019] For each pixel to be interpolated, an adaptive selection unit 460 selectively outputs 
either the pixel generated by the motion compensated interpolation unit 440 or the pixel 
generated by the spatio-temporal interpolation unit 450 according to the motion type (global 
motion or local motion) determined by the motion type determination unit 424 and the 
accuracy of the motion vector estimated by the motion estimation unit 410. 

[0020] A progressive frame formation unit 470 combines pixels selectively output from the 
adaptive selection unit 460 and the pixels of a current field to form a progressive image. 

[0021] FIG. 5 is a detailed flowchart of the operation of the motion analysis unit 422. In 
operation 510, a distribution of the motion vectors, which are estimated by the motion 
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estimation unit 410 with respect to adjacent fields, is estimated. Here, the distribution of 
motion vectors is represented using a histogram. 

[0022] In operation 520, values of the histogram of the motion vectors are compared with 
a fixed or variable threshold value. In operation 530, existence or non-existence of global 
motion in a current scene is determined based on a result of the comparison. Where a 
dominant peak exists in the histogram, there are many blocks having the same motion. 
Where a predetermined number of blocks have the same motion, the motion is estimated as 
global motion. 

[0023] FIG. 6A is a diagram illustrating motion vectors of a video image where global 
motion exists in a current scene. In FIG. 6A, the numbers on the horizontal and vertical 
axes of the diagram are indices of blocks in a range. FIG. 6B is a histogram of motion 
vectors in two adjacent homogeneous fields of the current scene. An interlaced format 
image may be referred to by reference to a top field and a bottom field. A homogeneous 
field refers to two fields of equal parity, that is, top filed and top field or bottom field and 
bottom field. The histogram shown in FIG. 6B indicates that there are many blocks having 
the same motion vector and a determination is made that global motion exists in an image in 
consecutive fields. The numbers on the vertical axis represent a number of occurrences of 
block motion vectors having a given magnitude and direction as indicated on the horizontal 
axis. The determination is made based on a comparison of a peak number of occurrences 
with a threshold value Th. 

[0024] FIG. 6C is a diagram illustrating motion vectors of a video image where global 
motion does not exist in a current scene but the current scene has a local motion. As in 
FIG. 6A, the numbers on the horizontal and vertical axes of the diagram in FIG. 6C are 
indices of blocks in a range and the numbers on the vertical axis of FIG. 6D represent a 
number of occurrences of block motion vectors having a given magnitude and direction as 
indicated on the horizontal axis. A histogram shown in FIG. 6D represents that there is no 
motion vector having a number of occurrences greater than the threshold value T h . In other 
words, no global motion exists in a current field image, and all motion vectors are classified 
as local motion vectors. 

[0025] FIGS. 6D and 6D show histograms of motion vectors in one direction only. 
Similar histograms are generated in the other direction. The search range for motion 



vectors is set to a predetermined value, such as for example, -16 pixels to +16 pixels. 
Hence a component of a motion vector, either V x or V y of H = [V x , V y ]t, can have values in 
[-16, 16]. The horizontal axes of FIGS. 6B and 6D represent these values. The vertical 
axes of FIGS. 6B and 6D represent a number of times a component of the motion vector has 
the corresponding value. In short, the horizontal axis of each of FIGS. 6B and 6D 
represents the component of a motion vector and the vertical axis represents the number of 
occurrences. 

[0026] FIG. 7 is a detailed flowchart of operation of the motion type determination unit 
424. In operation 710, the motion vector of a unit block in a field to be interpolated is 
estimated. 

PJ [0027] Next, a property of the motion vector in a block is estimated. Here, where a 

O magnitude of the motion vector of the block in the field to be interpolated is small or zero, the 
J motion type of the block in the field to be interpolated is set to zero motion. In contrast, 

r y where the magnitude of the motion vector in the block in the field to be interpolated is at 

:fi least a predetermined value, the motion vector of the block is compared with a global motion 

L. vector obtained from a current scene in operation 720. The global motion vector is the 

py motion vector that occurs most often, as determined from the histograms discussed above 

^ with reference to FIGS. 6B and 6D. The motion vector with the most occurrences is used 

Q as the global motion vector. The motion type of the block in the field to be interpolated is 

^ determined in operation 730. Where the difference between the motion vector of the block 

in the field to be interpolated and the global motion vector is less than a predetermined 
threshold value, the motion type of the block is set to global motion. In contrast, where the 
difference between the block motion vector and the global motion vector exceeds the 
predetermined threshold value, the motion type of the block is set to local motion. 

[0028] FIG. 8 is a detailed flowchart of the operation of the adaptive selection unit 460. 
In operation 810, a motion type is determined according to input motion type information. In 
operation 820, a threshold value is selected according to the motion type as determined by 
the motion type determination unit 424. In operation 830, one of motion compensated 
interpolation and spatio-temporal interpolation is selected according to the motion type and 
the selected threshold value. 
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[0029] In more detail, where the motion type of a block in a field to be interpolated is 
global motion, and where the accuracy of the motion of the block exceeds a predetermined 
threshold, pixels generated according to motion compensated interpolation are selected as 
the interpolated pixels. Where the motion type of a block in a field to be interpolated is 
global motion, and where the accuracy of the motion of the block is less than the 
predetermined threshold, pixels generated according to spatio-temporal interpolation are 
selected as the interpolated pixels. 

[0030] Where the motion type of a block in a field to be interpolated is local motion, and 
where the accuracy of the motion of the block exceeds the predetermined threshold, pixels 
generated according to motion compensated interpolation are selected as the interpolated 
U pixels. Where the motion type of a block in a field to be interpolated is local motion, and 

9 where the accuracy of the motion of the block is less than the predetermined threshold, 
%1 pixels generated according to spatio-temporal interpolation are selected as the interpolated 

j! pixels. 

K [0031] Where the motion type of a block in a field to be interpolated is zero motion, and 

■ where the accuracy of the motion of the block exceeds the predetermined threshold, pixels 

S3 generated according to motion compensated interpolation are selected as the interpolated 

pixels. Where the motion type of a block in a field to be interpolated is zero motion, and 
B where the accuracy of the motion of the block is less than the predetermined threshold, 

^ pixels generated according to spatio-temporal interpolation are selected as the interpolated 

pixels. Here, the accuracy of motion and the predetermined threshold which is compared 

with the accuracy vary according to motion type. 

[0032] In summary, according to the present invention, either a motion compensated 
interpolation value or a spatio-temporal interpolation value is adaptively selected according 
to the degree of motion in an input image, thereby efficiently reducing flickering and artifacts. 

[0033] Although a few embodiments of the present invention have been shown and 
described, it would be appreciated by those skilled in the art that changes may be made in 
these embodiments without departing from the principles and spirit of the invention, the 
scope of which is defined in the claims and their equivalents. 
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